A common misapplication of statistical inference: Nuisance control with null-hypothesis significance tests

An alternative to null-hypothesis significance tests.

The statistic p(rep) estimates the probability of replicating an effect. It captures traditional publication criteria for signal-to-noise ratio, while avoiding parametric inference and the resulting Bayesian dilemma. In concert with effect size and replication intervals, p(rep) provides all of the information now used in evaluating research, while avoiding many of the pitfalls of traditional st...

What role should null-hypothesis significance tests have in statistical education and hypothesis falsification?

3 Arrigo, K.R. et al. (1998) Primary production in Southern Ocean waters. J. Geophys. Res. 103, 15587–15600 4 Karnovsky, N. et al. (2007) The impact and importance of production in polynyas to top-trophic predators: three case histories. In Polynyas (Smith, W.O., Jr,, and Barber, D.G., eds), pp. 391–410, Elsevier 5 Smith, W.O., Jr and Nelson, D.M. (1985) Phytoplankton bloom produced by a recedi...

Nearly Optimal Tests when a Nuisance Parameter is Present Under the Null Hypothesis∗

This paper considers nonstandard hypothesis testing problems that involve a nuisance parameter. We establish an upper bound on the weighted average power of all valid tests, and develop a numerical algorithm that determines a feasible test with power close to the bound. The approach is illustrated in six applications: inference about a linear regression coefficient when the sign of a control co...

Statistical Tests, Tests of Significance, and Tests of a Hypothesis Using Excel

Null hypothesis significance testing: a short tutorial

Although thoroughly criticized, null hypothesis significance testing (NHST) remains the statistical method of choice used to provide evidence for an effect, in biological, biomedical and social sciences. In this short tutorial, I first summarize the concepts behind the method, distinguishing test of significance (Fisher) and test of acceptance (Newman-Pearson) and point to common interpretation...